Object Tracking and Indexing in H.264/AVC Bitstream Domains

The use of surveillance video recording is mainly based on the activity of monitoring the areas in which the surveillance cameras are located. Consequently, one of the motivations of observing the footage from surveillance video is to locate and identify regions of interest (e.g. suspected moving objects) in the covered areas. The visual information of objects such as color, shape, texture and motion etc. enables users to easily navigate and browse regions of interest in the surveillance video data. However, this requires the semi-automatic or automatic analysis and description of visual features for surveillance video data, which is usually very challengeable and time consuming. To describe the visual information of the surveillance video contents in a structured way for fast and accurate retrieval over visual queries, MPEG has standardized a set of visual descriptors in forms of metadata schema. The MPEG-7 Visual descriptors, as Part 3 of MPEG-7 (Multimedia Content Description Interface, standardized under ISO/IEC 15938), provide the standardized descriptions of visual features that enable users to identify, categorize or filter the image or video data (MPEG57/N4358). The MPEG-7 Visual descriptors enable the structured descriptions of color, texture, shape, motion, location and face recognition features. Moreover, since they are described in low-level, the retrieval process can be relatively easier by comparing a given query with the pre-generated descriptions of MPEG-7 Visual descriptors in database. In surveillance video contents, MPEG-7 Visual descriptor enables the description of the characteristics of objects. The motion trajectory can be used to describe the behavior of objects. It can track and identify when and where the objects move. The motion information for an object enables the retrieval of the object by querying the intensity of the motion or the moving direction. The automatic extraction of visual features from video contents has also become an emerging issue with MPEG-7 Visual descriptors. Recently, H.264/AVC has become a popular video compression tool for video surveillance due to its high coding efficiency and the availability of its real-time encoding devices. By incorporating content analysis into the surveillance applications, intelligent surveillance can be possible for efficient searching and retrieval of surveillance video data. In this case, the content analysis for the surveillance video data is usually required to be performed in real-time for metadata generation. For real-time generation of metadata, the content analysis is usually performed in the bitstream domain of compressed video instead of its raw pixel domain. In this chapter, we explain a feature extraction of motion trajectory which detects and tracks moving objects (ROI’s) and